Method for identifying a cause of blockage in a sequence of images, a computer program for performing said method, a computer-readable recording medium containing such computer program, a driving assistance system capable of executing said method

ABSTRACT

A method for identifying a cause of blockage in a sequence of images provided by a camera of a vehicle, the method comprising iteratively:S10) acquiring an image of the camera; successively acquired images forming a sequence of images;S20) detecting a blockage in last images of the sequence of images;S60) determining whether it is day-time or night-time based on time information;if it is determined that it is night-time:S82) determining whether toggling front light(s) of the vehicle on/off causes a change in images acquired by the camera; andS84) in this case, determining that for the current iteration, the cause of the blockage is presumably the road being dark.A computer program for performing said method, a computer-readable recording medium containing such computer program, a driving assistance system capable of executing said method.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a National Stage of International Application No.PCT/EP2017/067157 filed Jul. 7, 2017.

TECHNICAL FIELD

The present invention is directed to a method for identifying a cause ofblockage in a sequence of images provided by a camera mounted in avehicle which is moving on a road.

BACKGROUND ART

Cameras have now become an indispensable equipment on most cars wherethey provide highly valuable information, in particular in automateddriving systems, for instance in advanced driver assistance systems(ADAS).

Although they are usually highly reliable, in different circumstancescameras can experience problems which lead to a situation known asblockage situation, where a field of view of the camera appears to beblocked.

In such a blockage situation, a part or possibly the whole image doesnot change or only very slightly changes over successive imageacquisitions by the camera.

While this situation can be normal, for instance if the scene viewed bythe camera does not change, conversely such blockage situation can alsobe caused by a failure of the camera or by a problem related to thecamera such as icing or fogging of the windscreen behind which thecamera is placed.

For this reason, at least for cameras mounted aboard vehicles, it isusually necessary to detect blockage situations of the camera(s) and toreact appropriately to this situation.

The problem of camera blockage has already been identified and methodshave been developed in order to automatically detect a blocked field ofview in a camera. Such a method is for instance disclosed by document US2010/0182450.

However, detecting that a camera has a blocked field of view is notsufficient to determine how to react to this situation. It does notexplain why the field of view is blocked, and does not either help tofix the problem so that the camera works again. Accordingly, there is aneed for a method providing the cause of a blockage of a camera, andthus making it possible to take action to get the camera to functionnormally again.

The invention has been constructed in view of the above problems of theprior art, and a first object of the invention is therefore to propose amethod for identifying the cause of a blockage of a camera.

DISCLOSURE OF THE INVENTION

According to the present invention, it is proposed a method foridentifying a cause of blockage in a sequence of images provided by acamera mounted in a vehicle which is moving on a road, the methodcomprising iteratively performing steps of:

S10) acquiring an image of the camera; successively acquired images thusforming said sequence of images;

S20) detecting a blockage in last images of the sequence of images;

S60) determining whether it is day-time or night-time based at least ontime information;

if it is determined that it is night-time, performing a step S80comprising:

S82) determining whether toggling front light(s) of the vehicle on/offcauses a change in images acquired by the camera; and

S84) if it is determined that toggling front light(s) on/off causes achange in said images acquired by the camera, determining that for thecurrent iteration, the cause of the blockage is presumably the roadbeing dark.

Advantageously, the simple determination of step S82 makes it possibleto determine that the cause of the blockage is presumably the road beingdark. This situation of a ‘dark road’ is simply a situation where thevehicle moves on a road which does not bear any marking, and isuniformly dark. Such situation can happen for instance just after a roadhas been renewed, before the lane markings have been applied.

Advantageously, this method uses information provided by several sensorsof the car, as well as (in some embodiments) information normally neededfor the guidance of the car, such as a database comprising records oflanes having lane markings, to identify the cause of the blockage of thecamera.

The method can further be adapted to also identify causes of blockageshappening during day trips.

Accordingly, in an embodiment the method further comprises, if, duringan iteration, it is determined at step S60 that it is day-time,performing a step S70 comprising:

S72) determining whether outside temperature is below a low temperaturethreshold; and

S73) if it is determined that the outside temperature is below the lowtemperature threshold, determining that for a current iteration, thecause of the blockage is presumably icing or fogging.

Thanks to this method, simply by determining that it is day-time (stepS60) and by determining that the outside temperature is below a lowtemperature threshold (step S73), it is possible to determine that thecause of the blockage is presumably icing or fogging.

An icing situation is a situation in which ice is formed on one of thewalls which separate the sensing surface of the camera from the scene itfilms (these walls are the walls of the windscreen or of the lens(es) ofthe camera), which ice causes the image to be blocked. Usually the icewould be formed on the windscreen itself.

A fogging situation is similar, except that instead of ice, fog isformed on the wall(s) separating the sensing surface of the camera fromthe scene.

The method is based on an algorithm which is executed iteratively on acomputer. The word ‘computer’ here must be construed broadly as anycomputing platform, including one or more electronic control unit(s),graphic processing unit(s) (GPUs), regular computer(s), which can belocated or not aboard the vehicle.

The method advantageously can be run in real time, but it is notnecessary. If the method is not run in real time, of course the time toconsider at step S60 is the time at which the last image of the sequenceof images was acquired.

In a variant of the above embodiment, the method further comprises atstep S70, if it has been determined at step S72 that the outsidetemperature is below the low temperature threshold, performing the stepsof:

S74) determining whether dew point is reached;

if it is determined that the dew point is not reached, determining thatfor the current iteration, the cause of the blockage is presumably icingor fogging with a first probability;

S75) if it is determined that the dew point is reached, determining thatfor the current iteration, the cause of the blockage is presumably icingor fogging with a second probability which is higher than the firstprobability.

Consequently, simply by checking that the dew point is reached at stepS74, it is possible to assess more precisely the probability for thecause of the blockage to be icing or fogging:

-   -   if the dew point is not reached, the probability for the cause        of the blockage to be icing or fogging is then set to the first        probability;    -   If the dew point is reached, the probability for the cause of        the blockage to be icing or fogging is set to the second        probability, which is higher than the first probability.

In a variant of the embodiment of the method comprising a step S70, stepS70 further comprises the step S78 of, if it is determined that theoutside temperature is above the low temperature threshold or, in caseit is checked whether the dew point is reached, the dew point is notreached, determining that for the current iteration, the cause of theblockage is presumably a sunset/sunrise situation or a uniform landscapesituation.

Advantageously, the simple determination of step S78 makes it possibleto determine a second cause of blockage, that is, to determine that thecause of the blockage is presumably either a sunset/sunrise situation,or a uniform landscape situation.

A sunset/sunrise situation is a situation in which the camera, which hasprobably been oriented towards the sun, is subject to blooming, wherebythe acquired images present a blockage during this situation.

A ‘uniform landscape’ situation is a situation in which the landscape isuniform, and more precisely uniform to the point that a situation ofblockage is detected in the sequence of images successively acquired.

The method is an iterative method. At each iteration, the algorithm canidentify a presumed cause of the blockage.

In some embodiments, improvements can be included in order to make surethat the method correctly identifies the cause(s) of the blockages ofthe camera, and consequently to increase the reliability of the method.

For instance, in an embodiment, the method further comprises, during aniteration, before performing step S60, performing steps of:

S40) detecting lane markings in a last image of the sequence of images;and

S45) if lane markings are detected in the last image, returning to stepS10 of image acquisition.

In this embodiment, in steps S40 and S45, it is checked whether lanemarkings can be identified in the image and, in the iteration underconsideration, the determination of a cause of the blockage at steps S60and S70 is not performed if lane markings are detected in the last image(the last image acquired at step S10).

Indeed, it the camera is able to detect lane markings, it is presumedthat the camera is not really blocked. Therefore if the lane markingsare detected, performing steps 60 and 70 of the method would mostprobably lead to an erroneous conclusion of a blockage of the camera.

In addition, this embodiment of the method can be improved as follows.

In a variant, the method for identifying the cause of the blockagefurther comprises, during an iteration, before performing step S40,performing steps of:

S30) determining whether a lane on which the vehicle moves has lanemarkings, based on position information of the vehicle and a databasecomprising records of lanes having lane markings; and

S35) if a lane on which the vehicle moves does not have lane markings,returning to step S10 of image acquisition.

Indeed, steps S30 and S35 can advantageously lead to interrupt thecurrent iteration and to start a new iteration (at step 10) byperforming the simple test that the lane has markings at step 30. Thesesteps S30 and S35 in particular make it possible to avoid detecting lanemarkings in the last image, which consumes far more computing power thanchecking whether the road lane has road markings.

Another improvement to increase the reliability of the method uses theenvironment sensors of the vehicle, that is, the sensors which arecapable of detecting objects around the vehicle (for example the radars,the lidar(s), . . . ).

In particular, in an embodiment, the method further comprises, during aniteration, before performing step S60, performing steps of:

S50) detecting whether there is an object on the road in front of thevehicle, based on information other than information derived from theimages; S55) if no object is detected on the road in front of thevehicle, returning to step S10 of image acquisition.

In that case, in the iteration under consideration, the blockage causeidentification steps S60 and 70 are only performed if it is detectedthat there is an object on the road in front of the vehicle but, despitethe presence of this object, a blockage situation in the sequence ofimages has been detected.

Conversely, if no object is detected, the situation of blockage of thecamera is not considered as being sufficiently confirmed; the iterationis aborted and the method is resumed at step S10.

The method presented so far only provides presumed causes of blockage ofthe camera.

In order to have a more reliable determination of the cause of blockageof the camera, it can be preferable to require that the methodconsistently indicates the same cause over several iterations toconclude that this cause is indeed the cause of the blockage of thecamera.

Accordingly, in an embodiment the method further comprises, when duringan iteration, a first cause of blockage has been detected, performingthe steps of:

S90) assessing whether a blockage has been detected at least for each oflast N1 iterations, and assessing whether the cause of the blockage hasbeen determined to be presumably said first cause for at least N2iterations during the last N1 iterations, N1 and N2 being predeterminednumbers, and, if a blockage has been detected at least for each of lastN1 iterations, and the cause of the blockage has been determined to bepresumably said first cause for at least N2 iterations during the lastN1 iterations, triggering an action based on the determination that thecause of the blockage is said first cause.

This action can be for instance switching ON or OFF the air-conditioningor the air heater, etc.

The number of iterations required in step S90 to reach the conclusionthat the camera blockage is of a certain type may depend on the type ofblockage. For instance, a higher number of determinations might berequired to conclude that a blockage is of type icing or fogging ratherthan to conclude that a blockage is of sunrise/sunset or uniformlandscape.

The proposed method can be applied either to whole images of the camera,or only to portions of images of the camera.

For instance, in an embodiment of the method, the images are partialimages which are part of larger images acquired by the camera.

In order to execute the method according to the invention, the methodpreferably includes periodically the steps of:

S10) acquiring outside temperature;

S12) acquiring time, and possibly the date;

S14) acquiring current position information of the vehicle using ageographical positioning system.

As an alternative, these values can be considered as being constantduring the whole trip or at least a considered period.

The geographical positioning system mentioned above, hereinafter calledthe ‘GPS’, may be any system which provides, or outputs, thegeographical position of the vehicle.

The GPS can be a usual satellite-based GPS, but can be any systemproviding the same information. For example, the geographical positionof a vehicle can be determined (or at least updated, if an initialposition is known) based on a high-definition map by analyzing theimages acquired by the camera and/or point clouds acquired by a lidar.

In addition, the method preferably includes a step of controlling,periodically or even at each iteration, that the vehicle is moving (Thetest can be done by comparing the speed of the vehicle to a minimumspeed, for instance 10 km/h).

The result of the method according to the present invention is aninformation which can help to determine how to react to the blockage ofthe camera. Therefore usually, once the cause of the blockage has beenidentified (at least presumed), or preferably confirmed at step S90, thecause of the blockage is normally transmitted to a vehicle controlsystem and/or to the driver of the vehicle.

Based on this information, it can be decided for instance to turn on oroff a heating or an air-conditioning system of the vehicle based on thecause of the blockage of the camera.

In particular, if the cause of the blockage is determined to be icing orfogging, the driver (or a control system of the vehicle) can decide toactivate a heater in order to heat the camera housing, a defogger inorder to remove fog from a lens of the camera or of the windscreen, etc.depending on the particulars of the situation.

In a particular implementation, the various steps of the method foridentifying a cause of blockage in a sequence of images are determinedby computer program instructions.

Accordingly, the invention also provides a computer program which isstored on a computer readable storage media, and which is suitable forbeing performed in a computer, the program including instructionsadapted to perform the steps of the method described above when it isrun on the computer.

The computer program may use any programming language, and be in theform of source code, object code, or code intermediate between sourcecode and object code, such as in a partially compiled form, or in anyother desirable form.

The invention also provides a computer-readable recording mediumincluding instructions of a computer program as mentioned above.

The recording medium may be an entity or device capable of storing theprogram. For example, the medium may comprise storage means, such as aread only memory (ROM), e.g. a compact disk (CD) ROM, or amicroelectronic circuit ROM, or indeed magnetic recording means, e.g. afloppy disk or a hard disk.

Alternatively, the recording medium may be an integrated circuit inwhich the program is incorporated, the circuit being adapted to executeor to be used in the execution of the method in question.

Another object of the present invention is to provide a drivingassistance system for a road vehicle, the driving assistance systemcomprising a camera and being capable of determining a cause or at leasta presumed cause of a blockage in a sequence of images provided by thecamera.

Here, a driving assistance system is defined as any system whichprovides information and/or controls which is or are useful for drivinga vehicle. In the present case, the driving assistance system is mountedor is to be mounted in a road vehicle such as a car, a truck, etc.

In order to perform its function, a driving assistance system normallycomprises at least one sensor, an electronic control unit, and one ormore feedback device(s) which transmit(s) information to the driver,and/or act(s) on control member(s) of the vehicle (for instance thesteering shaft, the brake, the accelerator pedal or the like) instead ofthe driver, in order to take part or all of the driving load off thedriver, at least during some driving periods.

A driving assistance system can be for instance an automated drivingsystem of level 1 or more as defined by SAE norm J3016. Such anautomated driving system is a motor vehicle driving automation systemthat is capable of performing part or all of the dynamic driving task(DDT) on a sustained basis.

The above-mentioned object of the present invention is met by a drivingassistance system, comprising an electronic control unit, a camera, anouter temperature sensor; wherein the electronic control unit, thecamera and the outer temperature sensor are configured to be mounted ina vehicle, and wherein the electronic control unit is configured toiteratively:

S10) acquire an image of the camera; successively acquired images thusforming said sequence of images; S20) detect a blockage situation inlast images of the sequence of images;

S60) determine whether it is day-time or night-time based at least ontime information; and

if, during an iteration, the electronic control unit determines that itis night-time:

S82) determine whether toggling front light(s) of the vehicle on/offcauses contrast change in the images; and

S84) if the electronic control unit has determined that toggling frontlight(s) on/off causes contrast change in the images, determine that forthe current iteration, the cause of the blockage is presumably the roadbeing dark.

In an embodiment of the driving assistance system, the electroniccontrol unit is further configured, if, during an iteration, theelectronic control unit has determined that it is day-time:

S72) to determine whether outside temperature is below a low temperaturethreshold, based on information provided by the outer temperaturesensor; and

S73) if the electronic control unit has determined that the outsidetemperature is below a low temperature threshold, to determine that fora current iteration, the cause of the blockage is presumably icing orfogging.

In a variant of the above embodiment, the driving assistance systemfurther comprises a humidity sensor, and the electronic control unit isfurther configured (at step S70):

S74) to determine whether dew point is reached;

if the electronic control unit has determined that the outsidetemperature is below the low temperature threshold and that the dewpoint is not reached, to determine that for the current iteration, thecause of the blockage is presumably icing or fogging with a firstprobability; and

S75) if the electronic control unit has determined that the outsidetemperature is below the low temperature threshold and that the dewpoint is reached, to determine that for the current iteration, the causeof the blockage is presumably icing or fogging with a second probabilitywhich is higher than the first probability.

In a variant of the above embodiment of the driving assistance systemcomprising features S72 and S73, the electronic control unit is furtherconfigured:

S78) if the electronic control unit has determined that the outsidetemperature is above a low temperature threshold or dew point is notreached, to determine that for a current iteration, the cause of theblockage is presumably a sunset/sunrise situation or a uniform landscapesituation.

In an embodiment of the driving assistance system, the electroniccontrol unit is further configured, during an iteration, beforedetermining whether it is day-time or night-time:

S40) to detect lane markings in a last image of the sequence of images;and

S45) if at least one lane marking is detected in the last image, toreturn to step S10 of image acquisition.

In an embodiment of the driving assistance system, the electroniccontrol unit is further configured, during an iteration, beforedetermining whether it is day-time or night-time:

S50) to detect whether there is an object on the road in front of thevehicle, using environment information other than information derivedfrom the images; and

S55) if the electronic control unit has detected that there is an objecton the road in front of the vehicle, to return to step S10 of imageacquisition.

In an embodiment of the automated driving system, the electronic controlunit is further configured, when, during an iteration, a first cause ofblockage has been detected:

S90) to assess whether a blockage has been detected at least for each oflast N1 iterations, and to assess whether the cause of the blockage hasbeen determined to be presumably said first cause for at least N2iterations during the last N1 iterations, N1 and N2 being predeterminednumbers, and,

if a blockage has been detected at least for each of last N1 iterations,and the cause of the blockage has been determined to be presumably thefirst cause for at least N2 iterations during the last N1 iterations, totrigger an action based on the determination that the cause of theblockage is said first cause.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention may be better understood and its numerous otherobjects and advantages will become apparent to those skilled in the artby reference to the accompanying drawing wherein like reference numeralsrefer to like elements in the several figures and in which:

FIG. 1 is a figure showing four images acquired respectively in fourdistinct blockage situations of a camera;

FIG. 2 is a schematic front view of a vehicle equipped with a drivingassistance system in an embodiment of the invention;

FIG. 3 is a flowchart showing the steps of a method in a firstembodiment of the present invention;

FIG. 4 is a flowchart showing the steps of a variant of the methodillustrated by FIG. 3 , forming a second embodiment of the presentinvention; and

FIG. 5 is a schematic drawing showing the material architecture of thedriving assistance system of FIG. 2 .

DESCRIPTION OF THE PREFERRED EMBODIMENT

FIG. 2 shows a car 100 (an example of a vehicle) in which is mounted adriving assistance system 10 which forms an exemplary embodiment of thepresent invention.

The driving assistance system 10 (or, in short, the system 10) is, inthe present case, an automated driving system comprising an electroniccontrol unit 20 and several sensor units, namely a camera unit 30, alidar unit 32, an outer temperature sensor unit 34, a radar unit 36, aclose range sonar sensor unit 38, a GPS unit 40, a humidity sensor unit42. The locations and shapes of these components as shown on FIG. 1 arenot representative of the actual locations and shapes of the realcomponents. Each of the sensor units can comprise one or more sensors.For instance, the camera unit 30 can comprise one or more cameras, thelidar unit 32 can comprise one or more lidars, etc.

For sake of simplicity, in the present example it will be consideredthat the camera unit comprises only one camera, referenced camera 30.

Although system 10 comprises all the above-mentioned sensor units, theclaimed invention can be implemented with fewer sensor units, as definedin the claims.

The material structure of the driving assistance system 10 isillustrated by FIG. 4 .

System 10 comprises an electronic control unit 20—or ECU 20—, to whichall the above-mentioned sensor units (sensor units 30, 32, 34, 36, 38,40, 42) are connected.

The ECU 20 has the hardware architecture of a computer. The ECU 20comprises a microprocessor 22, a random access memory (RAM) 24, a readonly memory (ROM) 26, an interface 28. These hardware elements areoptionally shared with other units of the driving assistance system 10.The interface 28 comprises a driver interface with a (not-shown) displayto transmit information to the driver of the car 100, and interfaceconnections with actuators and other components of the car. Inparticular, interface 28 comprises a connection with headlights 44 ofthe car which makes it possible to turn the headlights on or off asdesired.

A computer program to identify a cause of blockage in a sequence ofimages acquired by camera 30 is stored in memory 26. This program, andthe memory 26, are examples respectively of a computer program and acomputer-readable recording medium pursuant to the invention.

The read-only memory 26 of the ECU 20 indeed constitutes a recordingmedium according to the invention, readable by the processor 22 and onwhich said program is recorded.

First Embodiment

The program stored in memory 26 includes instructions for executing thesteps of a first method for identifying a cause of blockage in asequence of images provided by camera 30, which constitutes a firstembodiment of the invention.

This method is now going to be described in reference to FIGS. 1, 2 and3 . As it will be explained in details below, this method can eitherprovide a presumed cause of a blockage of the camera, or can provide amore reliable indication on the cause of a blockage of the camera.

This method makes it possible to identify four different causes for ablockage of the camera, which are illustrated by FIG. 2 :

(1) A situation of icing, where ice formed on the windscreen (and/orpossibly even on a lens or on lenses of the camera) blurs the image andleads to detect a blockage of the camera;

(2) A situation of fogging, where fog formed on the windscreen (and/orpossibly even on a lens or on lenses of the camera) blurs the image andleads to detect a blockage of the camera;

(3) A situation of sunset/sunrise, or of uniform landscape; or

(4) A situation of dark road.

This method is an iterative method. The successive iterations areexecuted at regular intervals, for instance every 0.1 second.

At each iteration, several functions are executed, which correspond tothe corresponding steps of the method. Some of the steps are conditionalsteps, that is, are carried out only if conditions for carrying out thestep are fulfilled. In the present embodiment, all the steps of themethod are performed by the ECU 20. By executing these steps, the ECU 20identifies the cause of blockages which can happen in the imagesprovided by the camera.

The steps of the method are shown on FIG. 3 .

This method uses the following parameters:

Blockage counter ‘Tblock’ (integer), which counts the number ofiterations during which a blockage situation has been detected.

Day-time counter ‘K_Day’ (integer), which counts the number ofiterations during which the cause of the blockage has been determined asbeing presumably ‘Sunset/sunrise or uniform landscape’.

Night-time counter ‘K_Night’ (integer), which counts the number ofiterations during which the cause of the blockage has been determined asbeing presumably the road being dark (or ‘Dark road’).

Fog/Ice-time counter ‘K_Fog’ (integer), which counts the number ofiterations during which the cause of the blockage has been determined asbeing presumably icing or fogging.

S10—Image Acquisition

At step S10, an image outputted by the camera 30 is acquired by ECU 20.

Since at each iteration, an image of the camera 30 is acquired, the ECU20 successively acquires many images. These successively acquired imagesform a sequence of images. Each image is constituted of a matrix ofpixels, for instance having 800 columns and 600 lines, in a manner knownper se.

S20—Blockage Detection

At step S20, the ECU 20 detects a blockage in the sequence of images.The blockage is detected on the basis of the last images acquired by ECU20. The blockage can be detected using any available algorithm or methodfor detecting such a blockage (for instance the method described bydocument US 2010/0182450). The number of images used is selectedaccording to the method used for detecting the blockage.

If at step S20, a blockage is detected, the ECU 20 increments theblockage counter Tblock (step S25), and then the procedure continues atstep S30.

Conversely, if at step S20, no blockage has been detected, all countersTblock, K_Day, K_Night, K_Fog are reset to 0 (step S26), and then theprocedure is resumed at step S10.

S30, S35—Lane Markings Presence Detection

At step S30, the ECU 20 determines whether a lane on which the vehiclemoves has lane markings. The presence of lane markings is determinedbased on two items of information. The first item of information is theposition of the vehicle, acquired by the GPS unit 40.

The ROM 26 further comprises a database which includes records for allthe lanes of all the roads of the region in which car 100 moves.

Based on the position of the vehicle 100, the ECU 20 determines the laneon which the vehicle is moving, and then determines whether this lane(and in some cases, more precisely, this portion of the lane) has roadmarkings such as essentially white solid or dotted lines.

Step 35 is a conditional step. If the lane on which the vehicle movesdoes not have lane markings, at step S35 the iteration is aborted, andthe procedure is resumed at step S10.

Conversely, if the lane on which the vehicle moves has lane markings,the procedure then continues at step S40:

S40, S45—Lane Markings Detection

At step S40, the ECU 20 determines whether lane markings (at least onelane marking) can be detected in the last image of the sequence ofimages, that is, in the image acquired at step S10. The detection ofthese markings can be performed by any known image processing method.

Step 45 is a conditional step. At step 45, if at least one lane markingis detected in the last image, although a blockage has been detected atstep S20, it is presumed that the camera actually works correctly.Consequently, the current iteration is then aborted, and the procedureis resumed at step S10.

Conversely, if no lane marking is detected in the last image, it seemsto confirm that the camera 30 is blocked, and the procedure thereforecontinues at step S50.

S50, S55—Detection of an Object on the Road

At step S50, the ECU 20 determines whether there is an object on theroad in front of the vehicle. The object can be any object, but willprobably be in most cases a vehicle preceding car 100. It can also be abicycle, a motorbike, etc., or any object or objects present on theroad. The detection for step S50 is limited to objects (or parts ofobjects) which are or stand in the field of view of camera 30.

At step 50, this object or these objects are detected on the basis ofenvironment information provided by any of the environment sensors ofcar 100 except camera 30, or of any combination of these sensors.Environment information is information about the environment of thevehicle. Environment sensors are sensors which can detect the presenceof an object around the vehicle.

In the present case, the environment sensors of system 10 (except camera30) are the sensors of the lidar unit 32, of the radar unit 36, and/orof the close range sonar sensor unit 38; the objects around the car 100are detected by these environment sensors. More precisely, these objectsare detected by the ECU 20, based on environment information provided bythese sensors, that is, based on environment information other thanenvironment information derived from the images acquired by camera 30.

Step 55 is a conditional step. At step 55, if it is detected that thereis an object on the road in front of the vehicle, although a blockagehas been detected at step S20, it is presumed that the camera actuallyworks correctly. Consequently, the current iteration of the procedure isthen aborted, and the procedure is then resumed for a new iteration atstep S10.

Conversely, if no object is detected in the image, which seems toconfirm that the camera 30 is blocked, the procedure then continues atstep S60.

Note: Although steps 40,45 are performed before steps 50,55 in thisembodiment, they could be performed in the inverse order. Or, as analternative, only steps 40,45 could be performed, but not steps 50,55,or conversely only steps 50,55 but not steps 40,45. The invention couldalso be implemented without performing any of steps 40,45,50,55, but atthe cost of a reduced reliability of the method.

S60—Daytime Determination

At step S60, the ECU 20 determines whether it is day-time or night-time(at the time of acquisition of the last image of the sequence of images.In most cases, the method is executed in real time and the time ofacquisition of the last image of the sequence of images is simply thecurrent time for the vehicle).

To determine whether it is day-time or night-time, the ECU 20 uses timeinformation of the driving assistance system. The determination ofwhether it is night-time or day-time can be improved by taking intoaccount the date and/or the position of the vehicle (provided by the GPSunit 40), which influence the exact time of dawn and dusk.

In the ECU determines at step S60 that it is day-time, the procedurethen continues at step S70; otherwise, the ECU 20 determines that it isnight-time, and after step S60 the procedure continues at step S80.

S70—Presumed Blockage Cause Determination During Daytime

Step 70 is a conditional step. At step 70, the ECU 20 first performssteps S72 of determining whether the outside temperature is below a lowtemperature threshold, and whether the dew point is reached (that is,whether the air is saturated in water vapour, whereby any additionalvapour would condense). The outside temperature is measured by outertemperature sensor unit 34, which measures the temperature outside thevehicle. The humidity content of the atmosphere is measured by humiditysensor unit 42. Based on the outer temperature and the humidity contentof the atmosphere, the ECU 20 first determines whether the dew point ofwater is reached. If the dew point of water is reached, it can bepresumed that fogging has occurred on one of the transparent wallsthrough which the camera 30 sees. The ECU 20 also determines whether theouter temperature is negative or at least close to 0° C. It the outertemperature is negative or close to 0° C., it can be presumed that icinghas occurred on the windscreen or on a lens of camera 30, which causes ablockage to be detected.

In the present embodiment, if ECU 20 determines that the outsidetemperature is below or equal to 5° C. and the dew point is reached, ata step S73 ECU 20 determines that for the current iteration, the causeof the blockage is presumably icing or fogging (situation 1 or 2 on FIG.1 ), and increments counter K_Fog. The procedure then continues at step90.

Conversely, if at step S72 it is determined that the outside temperatureis above the low temperature threshold (5° C.) or that the dew point ofwater is not reached, the procedure continues at step S76.

At step S76, ECU 20 determines whether it is day-time or night-time.

If ECU 20 determines that it is night-time, no conclusion is reachedwith respect to the cause of the blockage detection; the currentiteration is aborted, and the procedure is resumed with a new iterationat step S10.

If conversely, ECU 20 determines that it is day-time, the procedurecontinues at step S78.

At step S78, ECU 20 determines that for the current iteration, the causeof the blockage is presumably a sunset/sunrise situation or a uniformlandscape situation (situation 3 on FIG. 1 ), and increments counterK_Day. The procedure then continues at step 90.

S80—Presumed Blockage Cause Determination During Night-Time

Step S80 is executed only when if it has been determined that it isnight-time, and accordingly when the headlights are on.

At step S80, the ECU 20 first determines in a step S82 whether togglingfront light(s) of the vehicle on/off causes contrast change in theimages.

Step 82 is carried out as follows.

The ECU 20 sends controls to turn the headlights 44 off during a veryshort period, and then to turn them on again.

During the period when the headlights are off, ECU 20 controls thecamera 30 to acquire at least one image. The ‘OFF’ images acquired bycamera 30 during this period are transmitted to ECU 20.

ECU 20 then controls camera 30 to acquire a few images after theheadlights 44 have been turned On. The ‘ON’ images acquired by camera 30during this latter period are also transmitted to ECU 20.

By comparing the OFF images to the ON images, the ECU 20 then determinesat a step S82, whether toggling the headlights (as an example of frontlight(s) of the vehicle) between on/off positions causes a change in theimages.

If at step S82, it is determined that toggling front light(s) on/offcauses contrast change in the images, at step S84 the ECU 20 determinesthat for the current iteration, the cause of the blockage is presumablythe road being dark, and increments counter K_Night. After step 84, theprocedure then continues at step S90.

Conversely, if it is not determined at step S82, that toggling theheadlights between on/off positions causes a change in the images, thecause of the blockage is not presumed to be the road being dark.Consequently, the procedure continues to step S70, in order to determinewhether the cause of blockage could be icing/fogging (situations 1 or 2on FIG. 1 ).

S90—Confirmation of Blockage Cause

The confirmation step 90 is executed each time it has been possible toidentify a presumed cause of the blockage.

In step 90, the ECU 20 tries to determine whether the cause of theblockage can now be considered as confirmed.

In this purpose, the ECU 20 checks the values of the different counters.

The ECU 20 first assesses whether a blockage has been detectedsuccessively during a sufficient number of iterations, for instance,during a minimum number of 6 iterations. ECU 20 accordingly checkswhether Tblock is at least equal to 6.

If this first requirement is satisfied, the ECU 20 then assesses whetherthe last-detected cause of blockage has been detected a sufficientnumber of times since situations of blockage have been detected. In thepresent embodiment, ECU 20 assesses whether the last-detected cause ofblockage has been detected at least 3 times, and therefore checkswhether one of the counters K_Fog, K_Day, or K_Night is at least equalto 3. The counter which is checked is the counter which corresponds tothe last cause of blockage that has been detected. The counters K_Fog,K_Day, or K_Night correspond respectively to three different causes ofblockage: icing/fogging (situations 1 or 2), sunrise/sunset or uniformlandscape (situation 3), or Dark road (situation 4).

Let us suppose for instance that the ECU 20 has just identified at stepS74 that the presumed cause of blockage is icing or fogging.

Accordingly at step S90, ECU checks if counter TBlock is at least equalto 6; if is it the case, ECU 20 then determines if the counter D_Fog isat least equal to 3.

If this is also the case, ECU 20 determines that the cause of theblockage is icing or fogging.

(Different values can be set for the threshold N2 for the variouscounters if it is judged that a specific cause of blockage require to bedetected a smaller or larger number of times before being sufficientlyconfirmed). If one of these counters fulfills this condition and is atleast equal to 3, the ECU determines that the cause of the blockage isof the type associated with that counter.

If the ECU 20 confirms that the cause of the blockage is icing orfogging, at step S110 the ECU automatically turns on theair-conditioning system of the car.

In another embodiment, the car is equipped with a heater for heating theatmosphere between the camera and the windscreen. In this embodiment, ifthe ECU 20 confirms that the cause of the blockage is icing or fogging,at step S110 the ECU automatically turns on said heater in order to heatthe atmosphere between the camera and the windscreen in order to deiceand/or defog the windscreen at this location.

Second Embodiment

A second method for identifying a cause of blockage in a sequence ofimages provided by camera 30, which constitutes a second embodiment ofthe invention, is now going to be described in reference to FIG. 4 .

This second method is identical to the first method except for step S70.Indeed in step S70, rather than carrying out in step S72 a double test(outside temperature and dew point) in a single step S72, these twotests are made successively.

Accordingly, step S70 is carried out as follows:

At step 72, the ECU 20 determines whether the outside temperature isbelow a low temperature threshold of 5° C. (but does not determinewhether the dew point is reached), based on the outside temperaturemeasured by outer temperature sensor unit 34.

At step S73, if the outer temperature is determined to be below or equalto 5° C., it is presumed that icing or fogging has occurred on thewindscreen or on a lens of camera 30, which causes a blockage to bedetected (situation 1 or 2 on FIG. 1 ). ECU 20 sets the probability Prof the blockage to be caused by icing or fogging to a first value P1,and increments the value of the counter K_Fog.

Then, at step S74, ECU 20 determines whether the dew point of water isreached, based on the humidity content of the atmosphere measured byhumidity sensor unit 42.

At step S75, if the dew point of water is reached, it is confirmed thatfogging has occurred on one of the transparent walls through which thecamera 30 sees. Consequently, ECU 20 increases the value Pr of theprobability that the blockage is being caused by icing or fogging, andsets this probability Pr to a value P2 higher than P1.

After step S75, the procedure continues at step 90.

In this case, when after step 90 the cause of the blockage is consideredas confirmed, different actions can be taken depending on theprobability Pr that the cause of the blockage is icing or fogging.

As in the first method, if at step S72, it is determined that theoutside temperature is above 5° C., the procedure continues at step S76,in which the ECU 20 determines that for the current iteration, the causeof the blockage is presumably a sunset/sunrise situation or a uniformlandscape situation (situation 3 on FIG. 1 ).

The various counters (Tbloc, K_Fog, K_Day, K_Night) are used as in thefirst method.

The invention claimed is:
 1. A method for identifying a cause ofblockage in a sequence of images provided by a camera mounted in avehicle which is moving on a road, the method comprising iterativelyperforming steps of: acquiring an image from the camera, successivelyacquired images thus forming said sequence of images; detecting ablockage in last images of the sequence of images; determining whetherit is day-time or night-time based at least on time information; basedon determining that it is night-time, performing further stepscomprising: turning off front lights of the vehicle for a first timeduration; acquiring, during the first time duration, at least one firstimage from the camera; turning on the front lights of the vehicle for asecond time duration; the second time duration immediately following thefirst time duration; acquiring, during the second time duration, atleast one second image from the camera; comparing the at least one firstimage with the at least one second image to determine whether togglingthe front lights of the vehicle causes a contrast change between the atleast one first image and the at least one second image; and based ondetermining that the toggling of the front lights causes the contrastchange between the at least one first image and the at least one secondimage, determining, that for a current iteration, the cause of theblockage is that the road is dark; and based on determining that it isday-time, performing further steps comprising: determining whether a dewpoint is reached; when it is determined that the dew point is notreached, determining that for the current iteration, the cause of theblockage is icing or fogging with a first probability; and when it isdetermined that the dew point is reached, determining that for thecurrent iteration, the cause of the blockage is icing or fogging with asecond probability which is higher than the first probability.
 2. Amethod for identifying a cause of blockage according to claim 1, whereinwhen it is determined that it is day-time, the method further comprises:determining whether an outside temperature is below a low temperaturethreshold; and when it is determined that the outside temperature isbelow the low temperature threshold, determining that for the currentiteration, the cause of the blockage is icing or fogging.
 3. A methodfor identifying a cause of blockage according to claim 2, wherein themethod further comprises: when it is determined that the outsidetemperature is above the low temperature threshold or a dew point is notreached, determining that for the current iteration, the cause of theblockage is a sunset/sunrise situation or a uniform landscape situation.4. A method for identifying a cause of blockage according to claim 1,wherein the method further comprises, during an iteration, beforedetermining whether it is day-time or night-time based at least on thetime information, performing steps of: detecting lane markings in a lastimage of the sequence of images; and when lane markings are detected inthe last image, returning to the step of image acquisition.
 5. A methodfor identifying a cause of blockage according to claim 4, wherein themethod further comprises, during the iteration, before the detecting ofthe lane markings in the last image of the sequence of images,performing steps of: determining whether a lane on which the vehiclemoves has the lane markings, based on position information of thevehicle and a database comprising records of lanes having the lanemarkings; and when the lane on which the vehicle moves does not have thelane markings, returning to the step of the image acquisition.
 6. Amethod for identifying a cause of blockage according to claim 1, whereinthe method further comprises, during an iteration, before determiningwhether it is day-time or night-time based at least on the timeinformation, performing steps of: detecting whether there is an objecton the road in front of the vehicle, based on information other thaninformation derived from the images; and when no object is detected onthe road in front of the vehicle, returning to the step of imageacquisition.
 7. A method for identifying a cause of blockage accordingto claim 1, wherein the method further comprises, when, during aniteration, a first cause of blockage has been detected, performing:assessing of whether a blockage has been detected at least for each oflast N1 iterations, and assessing whether the cause of the blockage hasbeen determined to be said first cause for at least N2 iterations duringthe last N1 iterations, N1 and N2 being predetermined numbers; and whena blockage has been detected at least for each of last N1 iterations,and the cause of the blockage has been determined to be said first causefor at least N2 iterations during the last N1 iterations, triggering anaction based on the determination that the cause of the blockage is saidfirst cause.
 8. A method for identifying a cause of blockage accordingto claim 1, wherein the images are partial images which are part oflarger images acquired by the camera.
 9. A computer program which isstored on a non-transitory computer readable storage media, and which issuitable for being performed on a computer, the computer programincluding instructions adapted to perform the steps of a methodaccording to claim 1 when it is run on the computer.
 10. Anon-transitory computer-readable recording medium including instructionsof a computer program according to claim
 9. 11. A driving assistancesystem, comprising an electronic control unit, a camera, an outertemperature sensor, the electronic control unit, the camera and theouter temperature sensor being configured to be mounted in a vehicle;wherein the electronic control unit is configured to iteratively:acquire an image from the camera, successively acquired images thusforming a sequence of images; detect a blockage situation in last imagesof the sequence of images; determine whether it is day-time ornight-time based at least on time information; and, based on, during acurrent iteration, the electronic control unit having determined that itis night-time: turn off front lights of the vehicle for a first timeduration; acquire, during the first time duration, at least one firstimage from the camera; turn on the front lights of the vehicle for asecond time duration; the second time duration immediately following thefirst time duration; acquire, during the second time duration, at leastone second image from the camera; compare the at least one first imagewith the at least one second image to determine whether toggling thefront lights of the vehicle causes a contrast change between the atleast one first image and the at least one second image; and based onthe electronic control unit having determined that the toggling of thefront lights causes the contrast change between the at least one firstimage and the at least one second image, determine, that for the currentiteration, the cause of the blockage is that the road is dark; and basedon, during the current iteration, the electronic control unit havingdetermined that it is day-time: determine whether a dew point isreached; when it is determined that the dew point is not reached,determine that for the current iteration, the cause of the blockage isicing or fogging with a first probability; and when it is determinedthat the dew point is reached, determine that for the current iteration,the cause of the blockage is icing or fogging with a second probabilitywhich is higher than the first probability.
 12. A driving assistancesystem according to claim 11, wherein the electronic control unit isfurther configured to, when, during an iteration, the electronic controlunit has determined that it is day-time: determine whether an outsidetemperature is below a low temperature threshold, based on informationprovided by the outer temperature sensor; and when the electroniccontrol unit has determined that the outside temperature is below thelow temperature threshold, determine that for the current iteration, thecause of the blockage is icing or fogging.
 13. A driving assistancesystem according to claim 12, wherein the electronic control unit isfurther configured to: when the electronic control unit has determinedthat the outside temperature is above the low temperature threshold ordew point is not reached, determine that for the current iteration, thecause of the blockage is a sunset/sunrise situation or a uniformlandscape situation.
 14. A driving assistance system according to claim11, wherein the electronic control unit is further configured to, duringan iteration, before determining whether it is day-time or night-time:detect lane markings in a last image of the sequence of images; and whenat least one lane marking is detected in the last image, to return toimage acquisition.
 15. A driving assistance system according to claim11, wherein the electronic control unit is further configured to, duringan iteration, before determining whether it is day-time or night-time:detect whether there is an object on the road in front of the vehicle,using environment information other than information derived from theimages; and when the electronic control unit has detected that there isan object on the road in front of the vehicle, to return to imageacquisition.
 16. A driving assistance system according to claim 11,wherein the electronic control unit is further configured to, when,during an iteration, a first cause of blockage has been detected: assesswhether a blockage has been detected at least for each of last N1iterations, and to assess whether the cause of the blockage has beendetermined to be said first cause for at least N2 iterations during thelast N1 iterations, N1 and N2 being predetermined numbers and when ablockage has been detected at least for each of last N1 iterations, andthe cause of the blockage has been determined to be the first cause forat least N2 iterations during the last N1 iterations, to trigger anaction based on the determination that the cause of the blockage is saidfirst cause.
 17. A method for identifying a cause of blockage in asequence of images provided by a camera mounted in a vehicle which ismoving on a road, the method comprising iteratively performing steps of:acquiring an image from the camera, successively acquired images thusforming said sequence of images; detecting a blockage in last images ofthe sequence of images; calculating a dusk time based at least one timeinformation and position information of the vehicle; determining whetherit is day-time or night-time based on the dusk time; determining whetherfront lights of the vehicle are turned on; based on determining that itis night-time and that the front lights of the vehicle are turned on,performing further steps comprising: turning off front lights of thevehicle for a first time duration; acquiring, during the first timeduration, at least one first image from the camera; turning on the frontlights of the vehicle for a second time duration; the second timeduration immediately following the first time duration; acquiring,during the second time duration, at least one second image from thecamera; comparing the at least one first image with the at least onesecond image to determine whether toggling the front lights of thevehicle causes a contrast change between the at least one first imageand the at least one second image; and based on determining that thetoggling of the front lights causes the contrast change between the atleast one first image and the at least one second image, determining,that for a current iteration, the cause of the blockage is that the roadis dark; and based on determining that it is day-time, performingfurther steps comprising: determining whether a dew point is reached;when it is determined that the dew point is not reached, determiningthat for the current iteration, the cause of the blockage is icing orfogging with a first probability; and when it is determined that the dewpoint is reached, determining that for the current iteration, the causeof the blockage is icing or fogging with a second probability which ishigher than the first probability.
 18. A method for identifying a causeof blockage according to claim 17, wherein the method further comprises,when, during an iteration, a first cause of blockage has been detected,performing: assessing of whether a blockage has been detected at leastfor each of last N1 iterations, and assessing whether the cause of theblockage has been determined to be said first cause for at least N2iterations during the last N1 iterations, N1 and N2 being predeterminednumbers; and when a blockage has been detected at least for each of lastN1 iterations, and the cause of the blockage has been determined to besaid first cause for at least N2 iterations during the last N1iterations, triggering an action based on the determination that thecause of the blockage is said first cause.
 19. A method for identifyinga cause of blockage in a sequence of images provided by a camera mountedin a vehicle which is moving on a road, the method comprisingiteratively performing steps of: acquiring an image from the camera,successively acquired images thus forming said sequence of images;detecting a blockage in last images of the sequence of images;determining whether it is day-time or night-time based at least on timeinformation; based on determining that it is night-time, performingfurther steps comprising: turning off front lights of the vehicle for afirst time duration; acquiring, during the first time duration, at leastone first image from the camera; turning on the front lights of thevehicle for a second time duration; the second time duration immediatelyfollowing the first time duration; acquiring, during the second timeduration, at least one second image from the camera; comparing the atleast one first image with the at least one second image to determinewhether toggling the front lights of the vehicle causes a contrastchange between the at least one first image and the at least one secondimage; based on determining that the toggling of the front lights causesthe contrast change between the at least one first image and the atleast one second image, determining, that for a current iteration, thecause of the blockage is that the road is dark; and based on determiningthat it is day-time, performing further steps comprising: determiningwhether an outside temperature is below a low temperature threshold;when it is determined that the outside temperature is below the lowtemperature threshold, determining that for the current iteration, thecause of the blockage is icing or fogging; and when it is determinedthat the outside temperature is above the low temperature threshold or adew point is not reached, determining that for the current iteration,the cause of the blockage is a sunset/sunrise situation or a uniformlandscape situation.